Information processing system

ABSTRACT

An information processing system includes a video processing device that processes video data including a plurality of frames, and a transmission device that transmits video data, to be processed by the video processing device, to the connected video processing device. The video processing device stores some frames constituting the video data, and the transmission device stores the remaining frames of the video data not stored in the video processing device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/448,858, filed Mar. 3, 2017 which is claiming priority based onJapanese Patent Application No. 2016-042540 filed Mar. 4, 2016, thedisclosures of which are incorporated by reference herein in theirentireties.

TECHNICAL FIELD

The present invention relates to an information processing system, andin particular, to an information processing system for processing videodata.

BACKGROUND ART

As security needs increase, the scale of a video surveillance system isincreasing. Further, the definition of video is getting higher, and aperiod for storing video data is getting longer. Accordingly, the volumeof video data to be stored in a video management system is increasing atan accelerating pace. Further, along with automatic detection of personsor objects by analyzing surveillance camera video images, there is alsoa need to store a large amount of high-quality data used as study data.

In view of such a situation, storage in a local PC is difficult tosatisfy a capacity requirement for large-volume video data. Therefore, amethod of arranging video data on cloud storage, having an architecturein which a large capacity and high reliability are secured, isconsidered. It should be noted that cloud storage means a service oflending an external storage region (storage) on the Internet.

Patent Document 1: JP 2001-525636 A

In general, however, cloud storage has the following characteristicsunlike direct storage:

-   -   As it is provided at a remote site, it is not guaranteed that a        network communication band is secured sufficiently, whereby        communication delay becomes large; and    -   There are consistency constraints in order to secure a large        capacity and high reliability, so it is not guaranteed that        recorded data can be read at any time.

It should be noted that cloud storage or the like usually has astructure of blocking reading and writing of data until data isreproduced for backup to maintain consistency.

With such characteristics as described above, in the case of placingvideo data on cloud storage, there is a problem a necessary video datacannot be acquired immediately whereby video data cannot be viewed onthe local PC side in real time.

Specifically, the following problems are caused:

(1) When video distribution in an extremely large band is required likea case where video images of a plurality of cameras are viewedsimultaneously, video distribution cannot be performed due to a shortagein the network communication hand. Even in the case of performing videoanalysis, large-volume video data must be transmitted to an analysisserver, so it takes time to acquire video data;

(2) When attempting to perform special operation (jump, forward, orrewind) at the time of reproducing video images, response to anoperation is degraded due to a communication delay between the cloudstorage and a local PC (PC on which video images are to be reproduced);and

(3) When attempting to reproduce or analyze video images storedimmediately before such as several seconds to several tens of secondsbefore, the video images may not be acquired immediately due toconsistency constraints of the cloud storage.

As a technology of storing video data, there is a technology disclosedin JP 2001-525636 A (Patent Document 1), Patent Document 1 discloses amethod of dividing video data and storing the divided pieces of data indifferent files in order to reduce the load on a server device thatdistributes the video data. Specifically, in order to reduce the load tocarry out high-speed reproduction, compressed data is divided intoanchor picture frames (I frames) and predictive picture frames (P framesand B frames), which are stored in different files.

With such a method, however, there is a problem that till impossible toreproduce video data stably at a high speed as described above. In otherwords, in Patent Document 1, the load on the server is reduced onlybecause data required for high-speed reproduction is divided in advance.Therefore, there still are problems such as distribution inability or adelay due to a shortage of network communication band, lowering ofresponse to reproduction operation, and a delay in video capturing dueto consistency constraints of cloud storage.

Moreover, there is also problem that processing cannot be performedstably at a h speed not only in the case of reproducing video data butalso in the case of performing any processing on video data such asanalysis of video data.

SUMMARY

In view of the above, an exemplary object of the present invention is toprovide an information processing system capable of solving theabove-described problem, that is, a problem that video data cannot beprocessed stably at a high speed.

An information processing system, according to an exemplary aspect ofthe present invention, is configured to include

a video processing device that processes video data including aplurality of frames; and

a transmission device that transmits the video data, to be processed bythe video processing device, to the video processing device connected,wherein

the video processing device stores some frames constituting the videodata, and

the transmission device stores remaining frames, not stored in the videoprocessing device, of the video data.

Further, a video processing device, according to another exemplaryaspect of the present invention, is a video processing device configuredto process video data including a plurality of frames, wherein

the video processing device stores some frames constituting the videodata, and

the video processing device processes the stored some frames, andprocesses remaining frames constituting the video data not storedherein, upon receiving the remaining frames from a transmission devicethat stores the remaining frames.

Further, a non-transitory computer-readable medium storing a program,according to another exemplary aspect of the present invention,comprises instructions for causing a video processing device, configuredto process video data including a plurality of frames, to performoperations including,

storing some frames constituting the video data, processing the storedsome frames, and processing remaining frames constituting the video datanot stored in the video processing device, upon receiving the remainingframes from a transmission device that stores the remaining frames.

Further, a video data processing method, according to another exemplaryaspect of the present invention, is a video data processing method forprocessing video data including a plurality of frames by a videoprocessing device that stores some frames constituting the video data.The method includes

by the video processing device, processing the stored some frames, andprocessing remaining frames constituting the video data not stored inthe video processing device, upon receiving the remaining frames from atransmission device that stores the remaining frames.

As the present invention is configured as described above, video datacan be processed stably at a high speed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating a configuration of aninformation processing system according to a first exemplary embodimentof the present invention;

FIG. 2 illustrates a data structure of video data;

FIGS. 3(A) to 3(C) illustrate a data structure of video data;

FIG. 4 illustrates an exemplary table stored e frame information storageunit disclosed in FIG. 1;

FIG. 5 illustrates an exemplary table stored in the I frame storage unitdisclosed in FIG. 1;

FIG. 6 illustrates an exemplary table stored in the difference framestorage unit disclosed in FIG. 1;

FIG. 7 illustrates an exemplary table stored in the difference frametemporary storage unit disclosed in FIG. 1;

FIG. 8 is a flowchart illustrating an operation of the video capturingsystem disclosed in FIG. 1;

FIG. 9 is a flowchart illustrating an operation of the I framecompression unit disclosed in FIG. 1;

FIG. 10 is a flowchart illustrating an operation at the time ofreproducing video data by the local PC disclosed in FIG. 1;

FIG. 11 is a flowchart illustrating an operation of the framerestoration unit disclosed in FIG. 1;

FIG. 12 is a functional block diagram illustrating a configuration of aninformation processing system according to a second exemplary embodimentof the present invention; and

FIG. 13 is a block diagram illustrating a configuration of aninformation processing system according to a third exemplary embodimentof the present invention

EXEMPLARY EMBODIMENT First Exemplary Embodiment

A first exemplary embodiment of the present invention will be describedwith reference to FIGS. 1 to 11. FIG. 1 is a diagram for illustrating aconfiguration of an information processing system. FIG. 2 and FIGS. 3(A)to 3(C) illustrate data structures of video data. FIGS. 4 to 7illustrate exemplary tables stored in the information processing system.FIGS. 8 to 11 illustrate operations of the information processingsystem.

[Configuration]

As shown in FIG. 1, an information processing system of the presentembodiment includes a video capturing system 10, cloud storage 20, and alocal PC (personal computer) 30 that are connected over a network.

The video capturing system 10 has a surveillance camera 11. Thesurveillance camera 11 is a camera provided to a location wheresurveillance is desired, and captures video images. It should be notedthat the surveillance camera 11 is not limited to one used forsurveillance. It may be a camera to be used for other purposes, andcapturing any kind of video images.

Along with it, the local PC 30 described below is not limited to oneused for reproducing video data but may be a video processing devicethat performs other processes such as analysis of video data. This meansthat the information processing system of the present invention is asystem storing a large number of units of large-volume video data to bereproduced or analyzed by the local PC 30, such as a surveillance camerasystem, a video archive system, and a video distribution system.

The video capturing system 10 is also an information processing devicehaving an arithmetic unit and a storage unit. The video capturing system10 includes an encoding unit a frame type determination unit 13, adifference frame information transmission unit 14, and an I frameinformation transmission unit 15, which are constructed by thearithmetic unit executing a program.

The encoding unit 12 performs compression coding (encoding) video datacaptured by the surveillance camera 11 according to a compression codingstandard conforming to MPEG (Moving Picture Experts Group) standard. Itshould be noted that compression coding standards conforming to MPEGstandard include MPEG4 (ISO/IEC 14496, ITU-T H.264), MPEG2 (ISO/IEC13818), H.264 (ISO/IEC 14496-10), and the like.

Here, the structure of compression coding conforming to MPEG standardwill be described with reference to FIG. 2. In the compression codingmethod conforming to MPEG standard, a technique called inter-frameprediction is used. Inter-frame prediction is a technique of holdingonly difference information between frames to thereby reduce the amountof data.

When compression coding is performed, video data is configured of framesof I frames, P frames, and B frames as in the example of FIG. 2. Theconfiguration of FIG. 2 is an example. A configuration in which thenumbers of P frames and B frames between I frames are different, and aconfiguration in which the number of frames in a GOP (Group Of Picture)is different, are also acceptable.

I frame (intra-frame) is a frame that is a basis of inter-frameprediction, which is a frame on which only intra-frame compressioncoding is performed. P frame (predictive-frame) is a frame only havingdifference information between it and the previous frame. B frame(bi-directional-frame) is a frame only having difference informationbetween it and the previous and following frames. As described above, asthe P frame and the B frame are frames having only differenceinformation with respect to other frames, they have less data capacitythan I frame. This means that I frame is generally a frame having largerdata capacity than P frame and B frame.

In general, most of the capacity of a moving image to which MPEGcompression coding is applied is the data capacity of I frames. In amoving image containing typical motions, the percentage thereof is 40%.In a video containing fewer motions such as a surveillance camera video,the percentage thereof is much larger (50% to 70%). It should be notedthat a frame serving as the basis of inter-frame prediction H.264 iscalled IDR (Instantaneous Decoder Refresh).

The frame type determination unit 13 acquires video data, to whichcompression coding is applied, from the encoding unit 12, and determinesthe types of the respective frames constituting the video data. Then,according to the determination result, the respective frames aredistributed. In the present embodiment, it is determined whether each ofthe frames constituting the video data to which MPEG compression codingis applied is an I frame, a P frame or a B frame. Then, the framesdetermined to be I frames are distributed to the I frame informationtransmission unit 15, and the other frames, that is, the framesdetermined to be P frames and B frames, are distributed to thedifference frame information transmission unit 14.

The I frame information transmission unit 5 transmits the distributed Iframes and information of the I frames to the local PC 30 and storesthem therein. At this time, the frame information transmission unit 15also transmits, as the I frame information, the frame number and theframe type each frame.

Meanwhile, the difference frame information transmission unit 14transmits the distributed P frames and B frames and the information ofthe frames to the cloud storage 20, and stores them therein. Further,the difference frame information transmission unit 14 transmits theinformation of the P frames and the B frames to the local PC 30.

Specifically, to the cloud storage 20, the difference frame informationtransmission unit 14 transmits the frame number of each frame as theinformation of the P frame or the B frame, for example. On the otherhand, the difference frame information transmission unit 14 transmits,as information of the P frames or the B frames, the frame number and theframe type of each frame, the reference frame number, and the ID of thecloud storage 20 to which the frame is transmitted, to the local PC 30.Here, in the case where there are units of cloud storage 20 to which theframes are transmitted, each unit of cloud storage 20 is assigned withan ID.

Next, a configuration of the cloud storage 20 (transmission device) willbe described. The cloud storage 20 is a storage unit provided on thenetwork. The cloud storage 20 includes an arithmetic unit and a storageunit, and also a difference frame storage unit 21 formed in the storageunit. The cloud storage 20 has a function of storing frames andinformation of the frames and transmitting the frames to the local PC30, as described below, when the arithmetic unit executes a program.

Specifically, the cloud storage 20 receives P frames and B framestransmitted from the video capturing system 10, and stores them in thedifference frame storage unit 21. The cloud storage 20 also adds a newrecord in a difference frame storage table illustrated in FIG. 6, formedin the difference frame storage unit 21, and stores a storage path andthe frame number of the stored frame in a “difference frame storagepath” column and a “frame number” column.

Next, a configuration of the local PC 30 (video processing device) willbe described. The local PC 30 is a terminal mainly operated by a userwho monitors video data to reproduce video data, which is a typicalinformation processing device including an arithmetic unit, a storageunit, a display unit 38, and other units. The local PC 30 includes an Iframe compression unit 33, an frame restoration unit 34, a reproductiontime frame acquisition unit 36, and a decoding unit 37, which areconstructed by the arithmetic unit executing a program. The local PC 30also includes a frame information storage unit 31, an I frame storageunit 32, and a difference frame temporary storage unit 35, which areformed in the storage unit. It should be noted that the local PC 30 alsohas a function of storing frames and information of the frames asdescribed below, which is provided by the arithmetic unit executing aprogram.

Specifically, the local PC 30 receives an I frame transmitted from thevideo capturing system 10, and stores it in the I frame storage unit 32.The local PC 30 also adds a new record to the I frame storage table,illustrated in FIG. 5, formed in the I frame storage unit 32, and storesthe storage path and the frame number of the stored frame in the “Iframe storage path” column and the “frame number” column, respectively.At this time, “uncompressed” is set in the “I frame compression status”column and the “frame type after compression of I frame” column isblank.

Further, the local PC 30 stores information of each frame transmittedfrom the video capturing system 10, in the frame information table,illustrated in FIG. 4, formed in the frame information storage unit 31.Specifically, the local PC 30 adds a new record to the frame informationtable, and stores the notified frame number and the frame type inn the“frame number” column and the “frame type” column, respectively. Then,in the case of an I frame, the “reference frame number” column is blankbecause there is no frame to be referred to. Further, value of “stored”is put in the “frame storing status” column because the local PC 30 hasan I frame. The “cloud storage ID” column is blank. On the other hand,in the case of a P frame or a B frame, a value of a notified referenceframe number is put in the “reference frame number” column, and a valueof “not stored” is put in the “frame storing status” column. Further, anID of the notified cloud storage is stored in the “cloud storage ID”column.

Further, the I frame compression unit 33 of the local PC 30 performscompression coding (processing by the compression coding systemconforming to MPEG standard, for example) on the I frame stored in the Iframe storage unit 32. While the details of the compression coding willbe described in the description of operation, FIGS. 3(A) to 3(C)illustrate a state of a frame configuration at the time of compression.FIG. 3(A) illustrates video data, in which only I frames are assumed tobe stored in the I frame storage unit 32 as illustrated in FIG. 3(B).The I frames are further applied with compression coding as illustratedin FIG. 3(C). Consequently, some frames become P frames or B framesafter compression, while some frames remain as I frames. In this way, byfurther applying compression coding to I frames and storing them in theI frame storage unit, it is possible to reduce the volume of data to bestored in the local PC 30.

Then, when the compressed frame is still an frame, the I framecompression unit 33 changes the “I frame compression” column of the Iframe storage table in the I frame storage unit 32 illustrated in FIG. 5to “compressed”, and changes the “frame type after compression of Iframe” column to “I frame”. On the other hand, when the compressed framebecomes a P frame or a B frame, the I frame compression unit 33 changesthe “I frame compression” column of the I frame storage table in the Iframe storage unit 32 illustrated in FIG. 5 to “compressed” and changesthe “frame type after compression of I frame” column to “P frame” or “Bframe”.

When the reproduction time frame acquisition unit 36 of the local PC 30receives a video data reproduction operation from a user, thereproduction time frame acquisition unit 36 acquires information of theframe to be reproduced, from the frame information storage unit 31. Thismeans that the reproduction time frame acquisition unit 36 refers to theframe information table in the frame information storage unit 31illustrated in FIG. 4, and checks the frame of the frame section to bereproduced, and a reference frame required for decoding the frame. Then,the reproduction time frame acquisition unit 36 refers to the frameinformation table in the frame information storage unit 31 illustratedin FIG. 4 to check whether the frame type of the required frame is an Iframe, a P frame, or a B frame.

When the frame type of the frame to be reproduced is an I frame, thereproduction e frame acquisition unit 36 refers to the I frame storagetable illustrated in FIG. 5 and acquires an I frame from the I framestorage unit 32 of the local PC 30. At this time, if the I frame isapplied with compression coding, the I frame is restored from thecompressed state by the I frame restoration unit 34, whereby theoriginal I frame is acquired. It should be noted that restoration by theI frame restoration unit 34 will be described in detail in thedescription of operation.

On the other hand, when the frame type of the frame to be reproduced isa P frame or a B frame, the reproduction time frame acquisition unit 36refers to the frame information table illustrated in FIG. 4, and checkswhether the frame to be reproduced is held in the local PC 30, from theframe information storage unit 31. When the frame is not in the local PC30, the reproduction time frame acquisition unit 36 checks the cloudstorage ID from the frame information table, and acquires a P frame or aB frame from the cloud storage 20 of the acquired ID. At this time, thereproduction time frame acquisition unit 36 stores the acquired P frameor B frame in the difference frame temporary storage unit 5 in the localPC 30, and stores the path in the local PC 30 that is the storagedestination, in the difference frame temporary saving table illustratedin FIG. 7 in the difference frame temporary storage unit 35. On theother hand, when the frame is already in the local PC 30, thereproduction time frame acquisition unit 6 refers to the differenceframe temporary saving table illustrated in FIG. 7, and acquires theframe from the difference frame temporary storage unit 35.

Then, when the reproduction time frame acquisition unit 36 acquires allof the frames to be reproduced, the decoding unit 37 performs decodingon all of the frames. At this time, the reproduction time frameacquisition unit 36 transmits the frame of the frame section to bereproduced and a reference frame required for decoding the frame, to thedecoding unit 37 while assigning the frame number. It should be notedthat as a sequence header containing information of the image size andthe like, for example, is also required to reproduce the video data,metadata such as a sequence header is also transmitted to the decoderunit 37. The capacity of the metadata such as a sequence header issmall, so it may be stored in either a local side or a cloud side.

Then, the decoding unit 37 links the I frame, transmitted from thereproduction time frame acquisition unit 36, and the difference frame inthe order of the frame numbers, and performs decoding. The decoding unit37 reproduces the video data decoded in such a manner, and displays iton the display unit 38. When reproduction by the decoding unit 37 iscompleted, the difference frame that is not required any more is removedfrom the difference frame temporary storage unit 35. Thereby, capacityof the video data stored in the local PC 30 is reduced.

[Operation]

Next, operation of the information processing system described abovewill be described with reference to the flowcharts of FIGS. 8 to 11.First, a flow of processing at the time of storing video data will bedescribed with reference to FIG. 8.

Regarding the video data captured by e surveillance camera 11 andapplied with compression coding according to the MPEG standard, theframe type determination unit 13 of the video capturing system 10determines the frame type (I frame, B frame, or P frame) of each framein the video data (step S1 of FIG. 8) Then, as a result of thedetermination, an I frame (Yes at step S1 of FIG. 8) is transmitted bythe I frame information transmission unit 15 to the local PC 30. At thistime, the I frame information transmission unit 15 also transmits theframe number, assigned from the beginning of the frames, to the local PC30.

Then, the local PC stores the transmitted I frame in the I frame storageunit 32. Further, the local PC adds a new record to the I frame storagetable in the I frame storage unit 32 illustrated in FIG. 5, storesvalues in the “frame storage path” column and the “frame number” column,and puts “uncompressed” in the “I frame compression”. The “frame typeafter compression of I frame” is blank (step S2 of FIG. 8).

Then, the I frame information transmission unit 5 transmits the framenumber and the frame type of the I frame to the local PC 30. Uponreceiving it, the local PC 30 adds a new record to the frame informationtable in the frame information storage unit 31 illustrated in FIG. 4,and puts values in the “frame number” column and the “frame type”column. In the case of an I frame, as there is no other frame to bereferred to, the “reference number” column is blank. Further, a value of“stored” is put in the “frame storing status” column (step S3 of FIG.8).

Further, as a result of determining the frame type by the frame typedetermination unit 13, when the frame is a P frame or a B frame (No atstep S1 of FIG. 8), the difference frame information transmission unit14 transmits it to the cloud storage 20. At this time, the differenceframe information transmission unit 14 also transmits the frame number,assigned from the beginning of the frames, to the cloud storage 20.

Then, the cloud storage 20 stores the transmitted P frame or B frame inthe difference frame storage unit 21. Further, a new record is added tothe difference frame storage table in the difference frame storage unit21 illustrated in FIG. 6, and values are stored in the “frame storagepath” column and the “frame number” column (step S5 of FIG. 8).

Then, the difference frame information transmission unit 14 acquires areference frame of the P frame or the B frame referring to thecompressed video data, and transmits it together with the frame numberand the frame type to the local PC 30. Upon receiving it, the local PC30 adds a new record to the frame information table in the frameinformation storage unit 31 illustrated in FIG. 4, and puts values inthe “frame number” column,the “frame type” column, and the “referenceframe number” column. Further, the local PC 30 puts a value of “notstored” in the “frame storing status” column, and stores an ID of thecloud storage in which the data is stored, in the “cloud storage ID”column (step S6 of FIG. 8).

After step S3 or S6, the video capturing system 10 checks whether or notthere is a frame of the video data, applied with compression coding,that has not been transmitted to the local PC 30 or the cloud storage 20(step S4 of FIG. 8). When there is a frame not having been transmitted(Yes at step S4 of FIG. 8), the information processing system returns tostep S1, and repeats the processing described above (steps S1 to S6 ofFIG. 8).

Next, processing in the case of further compressing the I frame storedin the I frame storage unit 32 in the local PC 30 will be described withreference to FIG. 9. It should be noted that it is also possible to keepthe I frame in the I frame storage unit 32 as a still image (I frame)without applying compression processing. In that case, the processingdescribed below will not be performed.

First, the I frame compression unit 33 links the I frames stored in theI frame storage unit 32 of the local PC 30 in the order from the newestframe number, and with respect to the linked I frames, defines a frameto be kept as an I frame after compression. For example, I frames arelinked as illustrated in FIG. 3(B) and then, as illustrated in FIG.3(C), an I frame is kept each 15 frames to have a frame configuration of“IPBBPBB . . . ”. If there is almost no movement in the video, theinterval between the frames kept as I frames may be broadened. It shouldbe noted that the order of frames to which compression processing isapplied is set such that a frame to be referred to is applied withcompression processing in preference.

Then, the I frame compression unit 33 checks whether or not each of thelinked I frames is a frame to be kept as an I frame after compression(step S11 of FIG. 9). Regarding a frame to be kept as an I frame aftercompression (Yes at step S11 of FIG. 9), it is not necessary of performcompression coding particularly, so such a frame is kept as an I frame(step S12 of FIG. 9).

Then, the I frame compression unit 33 changes the “I frame compression”column, in the I frame storage table in the I frame storage unit 32illustrated in FIG. 5, to “compressed”, and the “frame type aftercompression of I frame” column” changed to “I frame” (step S13 of FIG.9).

As a result of checking whether or not the frame is kept as an I frameafter compression (step S11 of FIG. 9), if the frame becomes a P frameor a B frame after compression (No at step S11 of FIG. 9), the I framecompression unit 33 generates a P frame or a B frame (step S15 of FIG.9). Specifically, the I frame compression unit 33 performs decoding(inverse quantization, inverse discrete cosine transformation) on thereference frame, generates a reference image, calculates a differencevalue in dye information between the reference image and the targetimage, and generates a P frame or a B frame. At this time, informationrepresenting a frame that is referred to is also included in the data ofthe P frame or the B frame. It should be noted that the difference framegeneration method used in this step utilizes the scheme of MPEGcompression coding, whereby it is possible to perform compression usingan existing function.

Then, the I frame compression unit 33 changes the “I frame compression”column of the I frame storage table in the I frame storage unit 32illustrated in FIG. 5 to “compressed”, and changes the “frame type aftercompression of I frame” to “P frame” or “B frame” (step S16 of FIG. 9).

After step S13 or S16, if there is any frame to which compressionprocessing has not been applied among the I frames in the I framestorage unit 32 (Yes at step S14 of FIG. 9), the video capturing system10 returns to step S11, and repeats the processing described above(steps S11 to S16 of FIG. 8).

Next, a flow of processing at the time of reproducing video data by thelocal PC 30 will be described with reference to FIG. 10. In the case ofperforming video reproduction or special operation such as jumping orfast forwarding, when a user moves the seek bar on the videoreproduction screen of the local PC 30, the point where the user desiresto reproduce the video image is designated (step S21 of FIG. 10).

Then, the reproduction time frame acquisition unit 36 acquires a recordof the frame number of the designated point, from the frame informationtable in the frame information storage unit 31 illustrated in FIG. 4.Then, the reproduction time frame acquisition unit 36 acquires the framenumber to be referred to, from the “reference frame number” column inthe acquired record, and also acquires all of the records on the columnsof the respective reference frame numbers, from the frame informationtable (step S22 of FIG. 10).

Then, the reproduction time frame acquisition unit 36 refers to theframe information acquired as describe above, and checks whether theframe type is I frame, from the information in the “frame type” column(step S23 of FIG. 10). When the frame type is I frame (Yes at step S23of FIG. 10), the reproduction time frame acquisition unit 36 refers tothe “I frame compression” column from the I frame storage table in Iframe storage unit 32 illustrated in FIG. 5. When the “I frame videocompression” column shows “compressed”, the I frame restoration unit 34performs restoration of the linked I frame described below to obtain theoriginal I frame (step S24 of FIG. 10). When the “I frame videocompression” column shows “uncompressed”, the reproduction time frameacquisition unit 36 refers to the path stored in the “I frame storagepath” column of the I frame storage table in the I frame storage unit 32illustrated in FIG. 5, and acquires the frame data of the path as Iframe data as it is (step S24 of FIG. 10).

On the other hand, when the frame type of the frame information acquiredat step S22 is not I frame but P frame or B frame (No at step S23 ofFIG. 10), the reproduction time frame acquisition unit 36 refers to the“frame storing status” column of the frame information table in theframe information storage unit 31 illustrated in FIG. 4. Then, it ischecked whether or not the frame has been in the local PC 30. If it isin the local PC 30, the reproduction time frame acquisition unit 36acquires frame data from the difference frame temporary storage unit 35(step S25 of FIG. 10). If the frame is not in the local PC 30, thereproduction time frame acquisition unit 36 acquires frame data from thedifference frame storage unit 21 of the cloud storage 20 (step S25 ofFIG. 10). At this time, the reproduction time frame acquisition unit 36acquires a cloud storage ID from the “cloud storage ID” column of theframe information table in the frame information storage unit 31illustrated in FIG. 4, and acquires difference frame data from the cloudstorage 20 of the acquired ID. It should be noted that the differenceframe acquired from the cloud storage 20 is temporarily stored in thedifference frame temporary storage unit 35 in the local PC 30, and thepath is stored in the difference frame temporary saving tableillustrated in FIG. 7.

Thereafter, regarding the frame information acquired at step S22, ifthere is any frame data not having been acquired (Yes at step S26 ofFIG. 10), the reproduction time frame acquisition unit 36 returns tostep S23, and repeats the processing described above (steps S23 to S26of FIG. 10).

Then, after acquiring every frame data of the frame information acquiredat step S22 (No at step S26 of FIG. 10), the reproduction time frameacquisition unit 36 reproduces video data consisting of the frame data(step S27 of FIG. 10). Specifically, the reproduction time frameacquisition unit 36 transmits all of the frame data and the framenumbers to the decoding unit 37. Then, the decoding unit 37 links Iframes and difference frames (P frames and B frame) in the order of theframe number, applies decoding, and displays video images on the screenof the display unit 38.

Here, with reference to FIG. 11, processing to restore the I frameapplied with compression coding, by the I frame restoration unit 34 atstep S24 of the video data reproduction processing, will be described.First, the I frame restoration unit 34 checks whether or not the “frametype after compression of I frame” of the frame to be decoded in the Iframe storage unit 32 illustrated in FIG. 5 is “I frame” (step S31 ofFIG. 11).

Then, when the “frame type after compression of I frame” column shows “Iframe” (Yes at step S31 of FIG. 11), as it is still an I frame after thecompression processing, it is not necessary to perform decoding.Accordingly, the I frame restoration unit 34 acquires the I frame dataas it is (step S32 of FIG. 11). On the other hand, when the “frame typeafter compression of I frame” column shows “P frame” or “B frame” (No atstep S31 of FIG. 11), the I frame restoration unit 34 performs decoding,and generates the original image (image having dye information) (stepS34 of FIG. 1). Then, the I frame restoration unit 34 performsintra-frame compression coding on the generated original image data, andacquires I frame data (step S35 of FIG. 11).

Then, when there is a frame not having been decoded among the frames tobe decoded (Yes at step S33 of FIG. 11), the I frame restoration unit 34returns to step S31, and repeats the processing described above (stepsS31 to S35 of FIG. 11).

As described above, in the present invention, only I frames of the videocompressed in a manner such as MPEG are stored in the local PC 30, andthe other difference frames (P and B frames) are stored in the cloudstorage 20. Thereby, it is possible to significantly reduce thecommunication band at the time of reproducing the video, while reducingthe storage capacity of the local PC 30. As a result, it is possible toreproduce the video with good response, including special operation suchas jumping at the time of reproduction. Even in the case where the localPC 30 performs video analysis, it is possible to deliver a video frameof the designated time part to the local PC 30 without delay.

For example, in the case where data cannot be acquired immediately fromthe cloud due to consistency constraints of the cloud storage, itpossible to only reproduce I frames in the local PC. As such, it ispossible to perform simple video reproduction before completion ofacquisition of the data from the cloud, which enables flexiblereproduction.

Further, in the present invention, I frames held in the local PC 30 arelinked, and further, the frames are compressed using the MPEGcompression coding function and stored. Thereby, it is possible tofurther compress the storage capacity in the local PC 30. Particularly,by using the system for the surveillance camera video, as there are lessmovements in the video, the compression efficiency of the linked Iframes increases, whereby the amount of data can be reducedsignificantly.

Furthermore, only difference frames (P and B frames) exist on the cloudstorage 20, and the video cannot be restored and reproduced without Iframes. Therefore, this system can also be used as a countermeasureagainst information leakage of the cloud storage 20.

It should be noted that while description has been given above based onthe premise that the local PC 30 performs processing to reproduce videodata, the local PC 30 may perform different processing on video data.For example, the local PC 30 may analyze video data. Thereby, in thecase of performing simple analysis on the I frames stored in the localPC 30, analysis processing can be performed stably at a high speed.Further, it is also possible to perform analysis in more detail byacquiring difference frames from the cloud storage 20 when necessary.

Second Exemplary Embodiment

Next, a second exemplary embodiment of the present invention will bedescribed with reference to FIG. 12. FIG. 12 is a functional blockdiagram illustrating a configuration of an information processing systemof the present embodiment.

As illustrated in FIG. 12, an information processing system of thepresent embodiment has a configuration that is almost similar to that ofthe first exemplary embodiment, except for the point described below. Inthe information processing system of the present embodiment, the I framestorage unit 32, the I frame compression unit 33, the I framerestoration unit 34, provided to the local PC 30 as described in thefirst exemplary embodiment, are provided to the cloud storage 20, asdenoted by the reference numerals 22, 23, and 24. This means that the Iframe information transmission unit 15 of the video capturing system 10also stores I frames on the cloud storage 20.

In the present embodiment, the I frame temporary storage unit 39 isprovided to the local PC 30, and the reproduction time frame acquisitionunit 36 acquires only I frames from the cloud storage 20 in advance, andstores them in the I frame temporary storage unit 39. Thereby, only Iframes are transmitted to the local PC 30 first, whereby simplereproduction (reproduction of discontinuous video) can be made first nthe local PC 30. Then, the local PC 30 acquires difference frames forthe part that is desired to be viewed in detail from the cloud storage20, whereby the video can be reproduced in detail.

It should be noted that while the I frame restoration unit 24 isdisposed on the cloud storage 20 side in the configuration of FIG. 12,the I frame restoration unit 24 may be disposed on the local side inorder to reduce the communication traffic volume between the cloudstorage 20 and the local PC 30.

Third Exemplary Embodiment

Next, a third exemplary embodiment of the present invention will bedescribed with reference to FIG. 13. FIG. 13 is a block diagramillustrating a configuration of an information processing system of thepresent embodiment.

The information processing system of the present embodiment includes avideo processing device 100 that processes video data consisting of aplurality of frames, and a transmission device 200 that transmits videodata, to be processed by the video processing device 100, to theconnected video processing device 100.

Then, the video processing device 100 stores some frames 110 of thevideo data constituting the video data. On the other hand, thetransmission device 200 stores the remaining frames 210 of the videodata not stored in the video processing device 100.

According to such a configuration, the video processing device 100 onlystores some frames 110 of the video data, and the remaining frames 210are stored in the transmission device 200. Accordingly, the storagecapacity of the video processing device 100 can be reduced.

In addition, when processing to reproduce video data is performed by thevideo processing device 100, processing to reproduce the some frames 110stored in the video processing device 100 can be performed. Accordingly,there is no need to acquire the rest of the video data from theconnected transmission device 200, whereby processing can be performedstably at a high speed.

<Supplementary Notes>

The whole or part of the exemplary embodiments disclosed above can bedescribed as, but not limited to, the following supplementary notes.Hereinafter, the outlines of the configurations of an informationprocessing system, a video processing device, a program, and a videodata processing method, according to the present invention, will bedescribed. However, the present invention is not limited to theconfigurations described below.

(Supplementary Note 1)

An information processing system comprising:

a video processing device that processes video data including aplurality of frames; and

a transmission device that transmits the video data, to be processed bythe video processing device, to the video processing device connected,wherein

the video processing device stores some frames constituting the videodata, and

the transmission device stores remaining frames, not stored in the videoprocessing device, of the video data.

With the above-described configuration, the video processing device onlystores some frames of the video data, and the remaining frames arestored in the transmission device. Therefore, it is possible to suppressthe storage capacity of the video processing device. In addition, whenreproducing the video data by the video processing device, it ispossible to perform processing stably at a high speed by performingprocessing to reproduce the some frames stored in the video processingdevice, for example.

(Supplementary Note 2)

The information processing system according to supplementary note 1,wherein

each of the some frames constituting the video data, stored in the videoprocessing device, has a capacity larger than a capacity of each of theremaining frames constituting the video data.

With this configuration,as the frame having a larger capacity is storedin the video processing device, processing can be performed more stablyat a higher speed compared with the case of performing processing byreceiving such a frame from the transmission device.

(Supplementary Note 3)

The information processing system according to supplementary note 1 or2, wherein

the video data is encoded by compression coding, and

each of the frames stored in the video processing device is a frameserving as a basis of inter-frame prediction of the video data, and eachof the frames stored in the transmission device is a frame includingdifference information between the frames of the video data.

(Supplementary Note 4)

The information processing system according to any of supplementarynotes 1 to 3, wherein

the video data encoded by compression coding conforming to MPEGstandard, and

each of the frames stored in the video processing device is an I frameserving as a basis of inter-frame prediction of the video data, and eachof the frames stored in the transmission device is a P frame or a Bframe including difference information between the frames of the videodata.

Thereby, the video processing device is able to process I fracasesstored in the own device stably at a high speed.

(Supplementary Note 5)

The information processing system according to any of supplementarynotes 1 to 4, wherein

the video processing device stores the some frames constituting thevideo data in a state of being encoded by compression coding.

(Supplementary Note 6)

The information processing system according to supplementary note 5,wherein

the video processing device stores the some frames constituting thevideo data in a state of being encoded by compression coding conformingto MPEG standard.

Thereby, as the video processing device stores the some frames in afurther compressed state, it is possible to reduce the storage capacity.

(Supplementary Note 7)

A non-transitory computer-readable medium storing a program comprisinginstructions for causing a video processing device to perform operationsincluding, the video processing device being configured to process videodata including a plurality of frames,

storing some frames constituting the video data, processing the storedsome frames, and processing remaining frames constituting the videodata, not stored in the video processing device, upon receiving theremaining frames from a transmission device that stores the remainingframes.

(Supplementary Note 8)

The non-transitory computer-readable medium storing the program,according to supplementary note 7, further comprising instructions forcausing the video processing device to perform operations including,

storing, as each of the some frames constituting the video data, a framehaving a capacity larger than a capacity f each of the remaining framesconstituting the video data stored in the transmission device.

(Supplementary Note 9)

The non-transitory computer-readable medium storing the program,according to supplement note 7 or 8, wherein

the video data is encoded by compression coding,

each of the frames stored in the transmission device is a frameincluding difference information between the frames of the video data,and

the program further comprises instructions for causing the videoprocessing device to perform operations including storing a frameserving as a basis of inter-frame prediction of the video data, as eachof the some frames constituting the video data.

(Supplementary Note 10)

The non-transitory computer-readable medium storing the program,according to any of supplementary notes 7 to 9, wherein

the video data is encoded by compression coding conforming to MPEGstandard,

each of the frames stored in the transmission device is a P frame or a Bframe including difference information between the frames of the videodata, and

the program further comprises instructions for causing the videoprocessing device to perform operations including storing an I frameserving as a basis of inter-frame prediction of the video data, as eachof the some frames constituting the video data.

(Supplementary Note 11)

The non-transitory computer-readable medium storing the program,according to any of supplementary notes 7 to 10, further comprisinginstructions for causing the video processing device to performoperations including,

storing the some frames constituting the video data in a state of beingencoded by compression coding.

(Supplementary Note 12)

The non-transitory computer-readable medium storing the program,according to supplementary note 11, further comprising instructions forcausing the video processing device to perform operations including,

storing the some frames constituting the video data in a state of beingencoded by compression coding conforming to MPEG standard.

(Supplementary Note 13)

A video data processing method for processing video data including aplurality of frames by a video processing device that stores some framesconstituting the video data, the method comprising

by the video processing device, processing the stored some frames, andprocessing remaining frames constituting the video data not stored inthe video processing device, upon receiving the remaining frames from atransmission device that stores the remaining frames.

(Supplementary Note 14)

The video data processing method according to supplementary note 13,wherein

the video processing device stores, as each of the some framesconstituting the video data, a frame having a capacity larger than acapacity of each of the remaining frames constituting the video datastored in the transmission device.

(Supplementary Note 15)

The video data processing method according to supplementary note 13 or14, wherein

the video data is encoded by compression coding,

the video processing device stores a frame serving as a basis ofinter-frame prediction of the video data, as each of the some framesconstituting the video data, and

the transmission device stores a frame including difference informationbetween the frames of the video data.

(Supplementary Note 16)

The video data processing method according to any of supplementary notes13 to 15, wherein

the video data is encoded by compression coding conforming to MPEGstandard,

the video processing device stores an I frame serving as a basisinter-frame prediction of the video data, as each of the some framesconstituting the video data, and

the transmission device stores a P frame or a B frame includingdifference information between the frames of the video data.

(Supplementary Note 17)

The video data processing method according to any of supplementary notes13 to 16, wherein

the video processing device stores the some frames constituting thevideo data in a state of being encoded by compression coding.

(Supplementary Note 18)

The video data processing method according to supplementary note 17,wherein

the video processing device stores the some frame constituting the videodata in a state of being encoded by compression coding conforming toMPEG standard.

(Supplementary Note 19)

A video processing device that processes video data including aplurality of frames, wherein

the video processing device stores some frames constituting the videodata, and

the video processing device processes the stored some frames, andprocesses remaining frames constituting the video data not storedtherein, upon receiving the remaining frames from a transmission devicethat stores the remaining frames.

(Supplementary Note 20)

The video processing device according to supplementary note 19 wherein

the video processing device stores, as each of the some framesconstituting the video data, a frame having a capacity larger than acapacity of each of the remaining frames constituting the video datastored in the transmission device.

(Supplementary Note 21)

The video processing device according to supplementary note 19 or 20,wherein

the video data is encoded by compression coding,

each of the frames stored in the transmission device a frame includingdifference information between the frames of the video data, and

the video processing device stores a frame serving as a basis ofinter-frame prediction of the video data, as each of the some framesconstituting the video data.

(Supplementary Note 22)

The video processing device according to any of supplementary notes 9 to21, wherein

the video data is encoded by compression coding conforming MPEGstandard,

each of the frames stored in the transmission device is a P frame or a Bframe including difference information between the frames of the videodata, and

the video processing device stores an I frame serving as a basisinter-frame prediction of the video data, as each of the some framesconstituting the video data.

(Supplementary Note 23)

The video processing device according to any of supplementary notes 19to 22, wherein

the video processing device stores the some frames constituting thevideo data in a state of being encoded by compression coding.

(Supplementary Note 24)

The video processing device according to supplementary note 23, wherein

the video processing device stores the some frames constituting thevideo data in a state of being encoded by compression coding conformingto MPEG standard.

It should be noted that the program described above may be stored in astorage unit or on a computer-readable medium. For example, a medium isa portable medium such as a flexible disk, an optical disk, amagneto-optical disk, or a semiconductor memory.

While the present invention has been described with reference to theexemplary embodiments described above, the present invention is notlimited to the above-described embodiments. The form and details of thepresent invention can be changed within the scope of the presentinvention in various manners that can be understood by those skilled inthe art.

1. An information processing system comprising: a video processingdevice that processes video data; a transmission device communicablewith the video processing device via a communication network; and avideo acquisition system configured to: acquire video data; generatebase frames and first difference frames by performing compression codingon the acquired video data, the base frames being frames serving as abasis of inter-frame prediction, and the first difference frames beingframes including difference information between frames of the acquiredvideo data; transmit the base frames to the video processing device; andcause the first difference frames to be stored in the transmissiondevice while not causing the first difference frames to be stored in thevideo processing device, wherein the video processing device isconfigured to: generate second difference frames by performing secondcompression coding with use of inter-frame prediction on the base framesgenerated by the video acquisition system; store the generated seconddifference frames; and when processing the acquired video data:reproduce the base frames using the second difference frames; receivethe first difference frames from the transmission device; and reproducethe video data based on the base frames and the first difference frames.2. The information processing system according to claim 1, wherein thesecond compression coding includes: performing decoding on the baseframes on which intra-frame compression coding has been performed; andcalculating a difference between decoded base frames.
 3. The informationprocessing system according to claim 1, wherein the video processingdevice is further configured to when replaying the video data based onthe base frames and the first difference frames, replay the video databased only on the base frames reproduced by using the second differenceframes in a case where the first difference frames are not able to bereceived immediately from the transmission device.
 4. An informationprocessing method performed by a video processing device communicablewith a transmission device and a video acquisition system via acommunication network, the video acquisition system being configured to:acquire video data; generate base frames and first difference frames byperforming compression coding on the acquired video data, the baseframes being frames serving as a basis of inter-frame prediction, andthe first difference frames being frames including differenceinformation between frames of the acquired video data; transmit the baseframes to the video processing device; and cause the first differenceframes to be stored in the transmission device while not causing thefirst difference frames to be stored in the video processing device, theinformation processing method comprising: generating second differenceframes by performing second compression coding with use of inter-frameprediction on the base frames generated by the video acquisition system;storing the generated second difference frames; and when processing theacquired video data: reproducing the base frames using the seconddifference frames; receiving the first difference frames from thetransmission device; and reproducing the video data based on the baseframes and the first difference frames.
 5. The information processingmethod according to claim 4, wherein the second compression codingincludes: performing decoding on the base frames on which intra-framecompression coding has been performed; and calculating a differencebetween decoded base frames.
 6. The information processing methodaccording to claim 4, further comprising when replaying the video databased on the base frames and the first difference frames, replaying thevideo data based only on the base frames reproduced by using the seconddifference frames in a case where the first difference frames are notable to be received immediately from the transmission device.
 7. Anon-transitory computer-readable storage medium storing a programcomprising instructions that causes a video processing device to performan information processing method, the video processing device beingcommunicable with a transmission device and a video acquisition systemvia a communication network, the video acquisition system beingconfigured to: acquire video data; generate base frames and firstdifference frames by performing compression coding on the acquired videodata, the base frames being frames serving as a basis of inter-frameprediction, and the first difference frames being frames includingdifference information between frames of the acquired video data;transmit the base frames to the video processing device; and cause thefirst difference frames to be stored in the transmission device whilenot causing the first difference frames to be stored inn the videoprocessing device, wherein the information processing method includes:generating second difference frames by performing second compressioncoding with use of inter-frame prediction on the base frames generatedby the video acquisition system; storing the generated second differenceframes; and when processing the acquired video data: reproducing thebase frames using the second difference frames; receiving the firstdifference frames from the transmission device; and reproducing thevideo data based on the base frames and the first difference frames. 8.The storage medium according to claim 7, wherein the second compressioncoding includes: performing decoding on the base frames on whichintra-frame compression coding has been performed; and calculating adifference between decoded base frames.
 9. The storage medium accordingto claim 7, wherein information processing method further includes whenreplaying the video date based on the base frames and the firstdifference frames, replaying the video data based only on the baseframes reproduced by using the second difference frames in a case wherethe first difference frames are not able to be received immediately fromthe transmission device.